Progress toward all-digital medical imaging environments has substantially increased the speed at which large amounts of medical image information can be accessed and displayed to a radiologist. As used herein, radiologist generically refers to a medical professional that analyzes medical images and makes clinical determinations therefrom, it being understood that such person might be titled differently, or might have differing qualifications, depending on the country or locality of their particular medical environment. X-ray based imaging for breast cancer screening/diagnosis is a particularly important field that is experiencing such information-expanding technological progress.
In what is referenced herein as conventional x-ray mammography, a stationary x-ray source projects x-rays through a breast that is immobilized by compression against a breast platform, while a stationary x-ray detector on the opposite side of the breast platform captures the x-rays to form a projection x-ray image. In the United States, two views are typically taken of each breast, including a craniocaudal (CC) view, in which the breast platform is horizontal and the x-rays are projected in a head-to-toe direction, and a mediolateral oblique (MLO) view, in which the breast platform is tilted at an angle (usually about 45 degrees) and the x-rays are projected normal to the breast platform from an inner-upper direction toward a lower-outer direction. For each patient in this typical scenario, the radiologist then carefully examines four images (left CC, left MLO, right CC, right MLO) to detect what are often subtle abnormalities indicative of a cancerous condition, such as particular types of microcalcification clusters and spiculated densities.
Although conventional x-ray mammography is still one of the best methods for detecting early forms of breast cancer, and is the modality approved by the U.S. Food and Drug Administration (FDA) to screen for breast cancer in women who do not show symptoms of breast disease, it is still possible for cancers to be missed by the radiologist reviewing the conventional x-ray mammograms. For example, for breasts that are high in dense fibroglandular content as compared to fat content, which is common for younger and/or smaller-breasted patients, conventional x-ray mammograms often contain saturated bright areas that can obscure cancerous conditions.
For these and other reasons, substantial attention and technological development has been dedicated toward breast x-ray tomosynthesis, which is similar in many respects to conventional x-ray mammography except that, for any particular view such as the CC or MLO view, the x-ray source is no longer stationary, but instead rotates through a limited angle relative to the breast platform normal (e.g., −15 degrees to +15 degrees) while several projection images (e.g., 10-15 projection images) are acquired by the x-ray detector. The several projection images are then mathematically processed to yield a relatively high number (e.g., 40-60) of tomosynthesis reconstructed images, each corresponding to a different slice of breast tissue, which can then be examined by the radiologist. Whereas a particular cancerous lesion positioned within a region of dense fibroglandular tissue might have been obscured in a single conventional x-ray mammogram view, that lesion could be readily apparent within a set of tomosynthesis reconstructed images representative of individual slices through the dense fibroglandular tissue. Examples of breast x-ray tomosynthesis systems can be found in U.S. Pat. No. 5,872,828, U.S. Pat. No. 7,123,684, and U.S. Pat. No. 7,245,694, each of which is incorporated by reference herein.
In additional to dedicated breast tomosynthesis systems, at least one proposal has been made for a combined-modality system capable of both conventional x-ray mammogram and breast x-ray tomosynthesis capabilities. The parent company of the assignee of this patent specification, Hologic, Inc., of Bedford, Mass., has demonstrated at trade shows in this country a fused, multimode mammography/tomosynthesis system that takes either or both of conventional x-ray mammograms and tomosynthesis projection images, either in single or multiple compressions/immobilizations of the breast. With regard to x-ray radiation dosage concerns, progress continues toward lowering per-projection image dosage such that the total radiation dose per breast is comparable to that of a conventional x-ray mammogram, while still maintaining good signal-to-noise ratio in the tomosynthesis projection and/or tomosynthesis reconstructed images.
Computer-aided detection (CAD) refers to the use of computers to analyze medical images to detect anatomical abnormalities therein, and/or the use of computers to otherwise process image information in a manner that facilitates perception of the medical image information by a radiologist. Sometimes used interchangeably with the term computer-aided detection are the terms computer-aided diagnosis, computer-assisted diagnosis, or computer-assisted detection. In an abnormality detection context, a CAD algorithm usually identifies a preliminary set of candidate detections in a medical image and then selects which ones, if any, will qualify as actual CAD detections based on a variety of computed features associated with the candidate detections. The CAD results, i.e., the body of information associated with the operation of the CAD algorithm on the medical image, are most often communicated in the form of annotation maps comprising graphical annotations (CAD markers) overlaid on a diagnostic-quality or reduced-resolution version of the medical image, one CAD marker for each CAD detection. Substantial effort and attention has been directed to increasing the analysis capabilities of CAD systems, resulting in ever-increasing amounts of information that is available to the radiologist for review.
Thousands of CAD systems for conventional x-ray mammography are now installed worldwide, and are used to assist radiologists in the interpretation of millions of mammograms per year. X-ray mammography CAD systems are described, for example, in U.S. Pat. No. 5,729,620, U.S. Pat. No. 5,815,591, U.S. Pat. No. 5,917,929, U.S. Pat. No. 6,014,452, U.S. Pat. No. 6,075,879, U.S. Pat. No. 6,301,378, and U.S. Pat. No. 6,574,357, each of which is incorporated by reference herein. Application of CAD algorithms to one or more of tomosynthesis projection images and tomosynthesis reconstructed images for use in conjunction with breast x-ray tomosynthesis systems has been proposed in U.S. Pat. No. 6,748,044, and U.S. Pat. No. 7,218,766, each of which is incorporated by reference herein.
However, in progressing from conventional x-ray mammography to breast x-ray tomosynthesis imaging, practical issues arise with regard to the rising volume of data requiring review by the radiologist. Whereas there are usually just four conventional x-ray mammogram images per patient, there can be hundreds of tomosynthesis reconstructed image slices (e.g., 40-60 slices for each of the four views). As more visual information becomes available, an important challenge is to present such information to the radiologist effectively and efficiently such that screening for abnormalities can be done thoroughly and effectively, and yet in a reasonable time to be practical, and diagnostic assessment can also be facilitated. Of particular importance is the manner in which an image review workstation displays CAD markers to the radiologist for the large stack of tomosynthesis reconstructed images, it being desirable that the CAD markers not be overly obtrusive on their corresponding image while also not being readily overlooked. The visual solution should simultaneously resolve these goals in conjunction with other beneficial goals, such as limiting the number of attention-sapping off-image gadgets to a workable minimum. Even subtle differences involving a few saved eye movements or a few saved hand strokes, keystrokes, or mouse cursor movements can lead to substantially better efficiency, stamina, and/or accuracy on the part of the radiologist.
As used in this patent specification, the notation Mp refers to a conventional x-ray mammogram as captured by a digital flat-panel detector, as
As used in this patent specification, the notation Mp refers to a conventional x-ray mammogram as captured by a digital flat-panel detector, as digitized from a film-screen cassette detector, and/or as processed to prepare it for display to the radiologist or for storage. The notation Tp refers to a tomosynthesis projection image that is similarly two-dimensional but is taken at a respective tomosynthesis angle between the breast and the origin of the imaging x-rays (typically the focal spot of an x-ray tube), and also encompasses the image as acquired as well as the image after being processed for display or for some other use. The notation Tr refers a tomosynthesis reconstructed image that is mathematically reconstructed from images Tp represents a slice of the breast as it would appear in a projection x-ray image of that slice at any desired angle, not only at an angle used for Tp or Mp images. In addition, a Tr image can represent a slice that conforms to any desired surface such as a flat or curved plane. Moreover, the process of reconstructing Tr images can use Mp images in addition to using Tp images or instead of one or more Tp images. The terms Tp, Tr and Mp also encompasses information, in whatever form, that is sufficient to describe such an image for display, further processing, or storage. The images Mp, Tp and Tr typically are in digital form before being displayed, and can be defined by information identifying properties of each pixel in a two-dimensional array of pixels although other ways to describe the images can be used as well or instead. The pixel values typically relate to respective measured or estimated or computed responses to x-rays of corresponding volumes in the breast (voxels or columns of tissue). A Tr image can represent a thin slice of a breast, in which case it may consist of pixel values representing respective voxels (volume elements) of the breast that are in a single layer or a few layers, or a Tr image may represent a thicker slice of the breast, in which case the pixel values of the thick-slice Tr image are calculated using known techniques such as, without limitation, a normalized projection of the pixels of several contiguous thin-slice images onto an image plane, a MIP (maximum intensity projection), or some other way of combining the pixel values representing several thin-slice images. As a non-limiting example, a thin-slice Tr image can represent a 1 mm thick slice of the imaged breast and a thick-slice Tr image can represent a 5-20 mm thick slice of the breast. Thus, when a breast is compressed for x-ray imaging to a thickness of 5-6 cm, there can be 50-60 thin-slice Tr images and 3-12 thick-slice Tr images.
Described in this patent specification are methods, systems, and related computer program products for processing and displaying CAD results to a user (e.g., a radiologist) in conjunction with breast x-ray tomosynthesis data in a manner that resolves several issues facing the user during review of such a large data set, including the need to recognize the existence and locations of CAD markers on a relatively small number of Tr images among a relatively large stack of Tr images, while not having inordinate amounts of attention devoted to searching for those CAD markers, without having an inordinate number of off-image eye movements, and without an inordinate amount of Tr image content obscured by annotation content. For example, in one preferred embodiment, a plurality of Tr images representative of slices of a breast having selected orientations and thicknesses is received, and CAD information is received identifying a detection-containing one of the Tr images and a coordinate location thereon at which a CAD detection marker is to be displayed. An ordered sequence of the Tr images is displayed to the user such that the user is allowed to sequentially page therethrough, the ordered sequence including the detection-containing Tr image. The ordered sequence of Tr images further includes at least one Tr image nearby the detection-containing Tr image with respect to the ordered sequence, the nearby Tr image not being detection-containing with respect to the identified coordinate location. When the detection-containing Tr image is displayed, the CAD detection marker is displayed thereon at the identified coordinate location. When the nearby Tr image is displayed, a CAD proximity marker is displayed thereon at the identified coordinate location, the CAD proximity marker not itself being indicative of a CAD detection on the nearby Tr image, but rather for encouraging user attention toward the identified coordinate location of the detection-containing Tr image, for discouraging user oversight of the CAD detection marker thereon when paging therethrough. Preferably, the CAD proximity has a noticeably different size than the CAD detection marker.
For one preferred embodiment, a CAD proximity marker is displayed on each of a plurality of nearby Tr images that are in an immediate neighborhood of the detection-containing Tr image, and the CAD proximity markers are all of noticeably different size relative to each other and to the CAD detection marker. Navigating toward the detection-containing Tr image (e.g., by mouse click, scroll wheel turn, arrow key click, or other paging command), the user pages depthwise from a farthest of the nearby Tr images to a closest of the nearby Tr images, the CAD proximity markers varying in size with each successive image and paging command. In one preferred embodiment, the CAD proximity markers are chirped in size, for example largest to smallest from the farthest nearby Tr image to the closest nearby Tr image. Advantageously, due to keen second-order motion perception in human peripheral vision, the presence of the CAD finding and its location on the display screen is readily noticed and perceived by the radiologist, even when the radiologist has been focusing on a different part of the display screen. Moreover, in many cases, the radiologist may so notice and perceive the presence of the CAD finding, and then continue to track their depthwise paging progression toward the detection-containing Tr image in their peripheral vision even as they continue to focus on a different part of the display.
In other preferred embodiments, further information is conveyed to the user by the CAD proximity markers without additional screen clutter. In one example, a paging behavior of the user, such as the user's paging rate toward or away from the detection-containing Tr image, is detected and instantly used to vary the displayed size of the CAD proximity markers. Thus, for example, if the user is paging very rapidly through the stack of Tr images, thereby increasing the likelihood they might miss the CAD detection marker on the detection-containing Tr image, the CAD proximity markers are made larger and more visible to ensure that the presence of the CAD detection marker is perceived during the fast paging process. However, if the use is paging very slowly, the CAD proximity markers are made smaller and less obtrusive, because it is less likely that the presence of the CAD detection marker will be overlooked. Other useful variations to the size and/or shape of the CAD proximity markers for achieving useful yet non-obtrusive information conveyance, including those described further hereinbelow, are within the scope of the preferred embodiments.